Skip to content

Conversation

@chingizof
Copy link

Description:

This pull request resolves a ValueError when converting OpenNMT models to CTranslate2 using ct2-opennmt-py-converter. The issue was due to unsupported --self_attn_type scaled-dot-flash, with only scaled-dot being supported.

Solution:
Implemented a fix based on a suggestion from a discussion by vince62s on the OpenNMT forum, which successfully addresses the conversion issue.

Testing:
Verified the fix by converting models that previously triggered the error, ensuring the process now completes without issues.

Reference:

Solution inspired by a post on OpenNMT Forum.
This update should help users facing similar conversion problems.

@vince62s
Copy link
Member

Hi, my message on the forum was a hack not really a fix.
In fact a better fix is to replace (when applicable) "scaled-dot-flash" by "scaled-dot" because in onmt-py we added "scaled-dot-flash" to force decoding using flash attention but it has no real usage in ct2.

@vince62s
Copy link
Member

closing in favor of #1687

@vince62s vince62s closed this Apr 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants